With the advance of the technique made in image process, the presentation of visual effect has been gradually brought from two-dimensional plane into three-dimensional space. As regards an input image, processes of generating a three-dimensional image can be classified into two main categories. In a process of a first category where several video cameras are used, the video cameras are positioned at different viewing angles to photograph the same objects, thus obtaining a number of two-dimensional images. In this way, as for the object which is to be presented in three-dimensional space, a number of viewpoint images such as the two-dimensional images captured at different angles can have their image information combined to synthesize a multi-view three-dimensional stereoscopic image.
In a process of a second category where a single video camera is used, the single video camera is positioned at a fixed viewing angle to photograph objects, thus obtaining a single two-dimensional image. In addition, a depth image corresponding to the two-dimensional image is provided to carry distance information of each object in the two-dimensional image. From the depth image, it can be derived that which object is located in the front of the two-dimensional image, i.e., in the front of the frame, and, in contrast thereto, which object is located in the rear of the two-dimensional image, i.e., in the rear of the frame. Therefore, the contained information of the two-dimensional image and the depth image can also be used to synthesize a multi-view three-dimensional stereoscopic image.
As is mentioned above, a single two-dimensional image along with its depth image can result in the generation or synthesis of a multi-view three-dimensional stereoscopic image. In the course of synthesis, a number of viewpoint images are generated and converted into a final image for outputting. Based on the depth image, shifts of image pixels to a new viewing angle are constructed to generate a viewpoint image which a viewer can observe from that viewing angle. However, the generated viewpoint image is not certainly an image with complete, intact image information. In other words, there could be holes remained in some region of the viewpoint image, and objects in the viewpoint image have some of their parts lost.
Refer to both FIGS. 1A, 1B, 1C, 1D, and 1E. FIG. 1A is a schematic diagram showing an original viewpoint image when observed from a center position. FIGS. 1B and 1C are schematic diagrams each showing a shifted viewpoint image when observed from a left position. FIGS. 1D and 1E are schematic diagrams each showing a shifted viewpoint image when observed from a right position. Viewpoint images 10a, 10b, 10c, 10d, and 10e are indicative of five viewpoint images which a viewer can observe at different viewing angles. The viewpoint image 10a is referred to as a viewpoint image at a central viewing angle, a two-dimensional image which is input originally without having its image pixels shifted. The viewpoint images 10b and 10c each are a shifted viewpoint image at a left viewing angle, while the viewpoint images 10d and 10e at a right viewing angle. Objects 110 and 120 in the images are represented by a triangular pattern and a square pattern, respectively. The objects 110 and 120 are in front of a label 140 indicative of background. The objects 110 and 120 have their location spatially correlated with each other. The object 110 is referred to as a foreground object since it is closer to the viewer, and the object 120 is referred to as a background object since it is behind the object 110.
When the viewer moves toward his or her left-hand side, the images he or she will see are illustrated by the viewpoint images 10b and 10c. In the viewpoint image 10b, a hole region 130b as denoted by slash marks “I” is appeared on the left side of the object 110. The reason the holes remain in the generated images is that the original two-dimensional image does not contain the image information of the hole regions 130b and 130c. Each of the hole regions 130b and 130c is indicative of a shift in relation to its base which is the viewpoint image 10a in this example. This can be also known as parallax difference which is caused when the viewer moves his or her position. In this regard, the hole regions 130b and 130c are where the view should see behind the object 110 but their true image information are absent in the original two-dimensional image, with the result that the hole regions 130b and 130c are generated. Similarly, in the viewpoint image 10d, a hole region 130d as denoted by slash marks “/” is appeared on the right side of the object 110. In the viewpoint image 10e, a hole region 130e as denoted by slash marks “/” is appeared on the right side of the object 110.
In addition to generating holes on left and right sides in the left and right viewpoint images, among those viewpoint images shifted toward the same direction, a viewpoint image, if corresponding to a larger distance between its viewing angle and the central viewing angle, has a hole region more obvious or wider than another. For example, the viewpoint images 10b and 10c are both left viewpoint images. Between them, the viewpoint image 10b has a larger distance between its viewing angle and the central viewing angle, so that its hole region 130b is more obvious than the hole region 130c. This means that the viewpoint image 10b can be found therein more image information which is absent in the original two-dimensional image. Similar situation applies to the viewpoint images 10e and 10d. Between them, the viewpoint image 10e has a larger distance between its viewing angle and the central viewing angle, so that its hole region 130e is more obvious than the hole region 130d. 